“Hi, Magic Closet, Tell Me What to Wear!”

In this paper, we aim at a practical system, magic closet, for automatic occasion-oriented clothing recommendation. Giv-en a user-input occasion, e.g., wedding, shopping or dating, magic closet intelligently suggests the most suitable clothing from the user’s own clothing photo album, or automatical-ly pairs the user-specified reference clothing (upper-body or lower-body) with the most suitable one from online shops. Two key criteria are explicitly considered for the mag-ic closet system. One criterion is to wear properly, e.g., compared to suit pants, it is more decent to wear a cock-tail dress for a banquet occasion. The other criterion is to wear aesthetically, e.g., a red T-shirt matches better white pants than green pants. To narrow the semantic gap be-tween the low-level features of clothing and the high-level occasion categories, we propose to adopt middle-level cloth-ing attributes (e.g., clothing category, color, pattern) as a bridge. More specifically, the clothing attributes are treated as latent variables in our proposed latent Support Vector Machine (SVM) based recommendation model. The wear-ing properly criterion is described in the model through a feature-occasion potential and an attribute-occasion poten-tial, while the wearing aesthetically criterion is expressed by an attribute-attribute potential. To learn a generalize-well model and comprehensively evaluate it, we collect a large clothing What-to-Wear (WoW) dataset, and thorough-ly annotate the whole dataset with 7 multi-value clothing attributes and 10 occasion categories via Amazon Mechanic Turk. Extensive experiments on the WoW dataset demon-strate the effectiveness of the magic closet system for both occasion-oriented clothing recommendation and pairing